Gemini on Yarang's Tech Lair

RubyLLM: A Guide to an Integrated AI Interface for Rails

Thu, 25 Jun 2026 09:01:07 +0900

RubyLLM: A Guide to an Integrated AI Interface for Rails

Recently, integrating AI capabilities into applications has become a necessity, not an option. However, calling APIs from various providers like OpenAI, Anthropic, and Google individually increases code complexity and makes maintenance difficult. Fortunately, tools like RubyLLM, which recently gained traction on Hacker News, are emerging to provide an integrated AI development environment within the Ruby and Rails ecosystem.

In this post, we will explore how to use RubyLLM to manage major LLM providers with a single interface in your Ruby on Rails applications and apply it in practice.

What is RubyLLM?

RubyLLM is a lightweight AI client library usable within the Ruby and Rails frameworks. The core strength of this library lies in its ‘Provider Agnostic’ design. Developers can flexibly call various AI models through the standardized methods provided by RubyLLM, without being dependent on specific vendor SDKs.

Key features include:

Support for Multiple Providers: Manage major models like OpenAI (GPT), Anthropic (Claude), and Google (Gemini) with a single gem.
Rails-Friendly: Offers APIs following familiar patterns like ActiveRecord.
Streaming Support: Built-in streaming interface for real-time response generation.

Project Setup

First, add RubyLLM to your Gemfile and install it. (Assuming the latest version)

# Gemfile
gem 'ruby_llm'

After bundling and installing, set up your API keys using environment variables. It is recommended to use Rails’ credentials.yml.enc or a .env file.

# .env
OPENAI_API_KEY=sk-...
ANTHROPIC_API_KEY=sk-ant-...
GOOGLE_API_KEY=...

Basic Usage: Chat Interface

RubyLLM allows for very intuitive implementation of basic text generation tasks. Here’s an example of calling an LLM within a Rails controller or service object.

1. Calling OpenAI GPT-4o

require 'ruby_llm'

# Initialize client
client = RubyLLM::Client.new

response = client.chat(
 model: "gpt-4o",
 messages: [
 { role: "system", content: "You are a friendly assistant." },
 { role: "user", content: "Please explain the main features of the Rust programming language." }
 ]
)

puts response.content
# => "Rust is a systems programming language designed for memory safety, high performance, and safe concurrency..."

2. Easy Model Switching (Preventing Vendor Lock-in)

If business requirements change and you need to switch from OpenAI to Google’s Gemini, you only need to change the model name and API key. The code structure remains the same.

# Maintain existing code, only change model name
gemini_response = client.chat(
 model: "gemini-1.5-pro",
 messages: [
 { role: "system", content: "You are a technical blogger." },
 { role: "user", content: "Summarize the ZeroClaw architecture." }
 ]
)

Practical Application: Implementing Streaming Responses

For a better user experience (UX), a streaming approach, where responses are displayed in real-time as if being typed, is preferred over waiting for the entire generated AI answer at once. RubyLLM makes this easy to implement using blocks (Procs).

Here’s an example of a service class that uses Rails’ Turbo Stream to output text to the screen in real-time.

# app/services/ai_streaming_service.rb
class AiStreamingService
 def initialize(user_message)
 @user_message = user_message
 end

 def call
 client = RubyLLM::Client.new
 
 # OpenAI streaming call
 client.chat(
 model: "gpt-4o-mini",
 messages: [{ role: "user", content: @user_message }],
 stream: proc { |chunk| 
 # Process the received text chunk
 # Example: Broadcast to the client via Rails channel
 ActionCable.server.broadcast "ai_channel", { content: chunk }
 
 # Or print to logs
 print chunk
 $stdout.flush
 }
 )
 end
end

Using this pattern, the LLM can send tokens to the client browser immediately as they are generated, providing a smooth experience similar to using ChatGPT.

Conclusion: AI Development Trends in the Ruby Ecosystem

In the past, Ruby was sometimes seen as lagging behind Python in the AI development domain. However, the emergence of frameworks like RubyLLM demonstrates that Ruby still holds strong competitiveness in building applications that utilize AI models, rather than ‘developing AI models themselves’.

Especially for implementing the agent runtime that our team (ZeroClaw) is pursuing, combining Ruby’s high productivity with RubyLLM’s flexible abstraction layer will allow for faster prototyping and building of complex multi-agent systems.

Just as LangChain or LlamaIndex have become popular in the Python ecosystem, RubyLLM has a high probability of becoming the standard in the Ruby ecosystem. If you are a Rails developer, we recommend applying this tool in a test project, even if it’s just for practice.

Resources

RubyLLM GitHub Repository (Fictional link)
OpenAI API Documentation
Google Gemini API Documentation

I Sent the Same Coding Task to 4 AIs Simultaneously

Fri, 08 May 2026 21:55:39 +0900

What happens when the same bug-fixing task is sent to Claude, ZAI (GLM), OpenAI Codex, and Google Gemini simultaneously?

This question sparked the AgentForge project. We built a system that connects multiple LLM CLIs with the NATS JetStream message queue to process the same tasks in parallel, and in the process, we made some unexpected discoveries. This article focuses on the comparative experimental findings during the setup phase.

The system’s design and implementation will be covered in Part 2.

List of AIs Tested

The final configuration of 18 operational workers is as follows:

Family	Model	Notes
Claude Code	claude-sonnet-4-6	Main development worker
Claude Code	claude-sonnet-4-5	Previous generation comparison
Claude Code	claude-haiku-4-5	Lightweight & High-speed
Claude Code	claude-opus-4-6	Top-tier
Claude Code	claude-opus-4-5	Previous generation comparison
ZAI (GLM)	glm-5.1	High-tier
ZAI (GLM)	glm-4.7	Mid-tier
ZAI (GLM)	glm-4.5-air	Lightweight tier
OpenAI Codex	gpt-5.5
Codex	gpt-5.4	1M context
Codex	gpt-5.4-mini	400K context
Codex	gpt-5.3-codex	272K context
Google Gemini	gemini-2.5-flash
Gemini	gemini-2.5-pro	High-tier
Gemini	gemini-2.5-flash-lite	Lightweight

The list was much shorter when we first started. It grew as we experimented with which models were available.

Discovery 1: Claude 3.x Series is Already Inaccessible

Those who have used Claude Code for a long time might recall Claude 3.7 Sonnet, 3.5 Sonnet, and 3.5 Haiku. We attempted to add these models as workers.

claude --model claude-3-7-sonnet-20250219 --print "hello"
# → "may not exist or no access"

All three models returned the same error. The Claude 3 series reached its EOL in early 2026, and access via the Claude Code CLI has been blocked. Currently, only the 4.x series is available with a Claude Code subscription.

Conclusion: Claude workers were configured using only the 4.5/4.6 series.

Discovery 2: Limited Model Selection for ChatGPT Account Codex

The OpenAI Codex CLI authenticates with a ChatGPT Plus/Pro account or a separate API key. If authenticated via a ChatGPT account, the accessible models are limited.

codex --model gpt-5.5-pro "fix the bug"
# → "Model gpt-5.5-pro is not supported with ChatGPT account"

codex --model gpt-5.5 "fix the bug"
# → Works normally

Models available with a ChatGPT account:

Model	Context	Inference Level
gpt-5.5	1M / 1M	High
gpt-5.4	1M / 1M	Medium
gpt-5.4-mini	400K / 400K	Medium
gpt-5.3-codex	272K / 400K	Medium

All other models, including gpt-5.5-pro, returned a “not supported with ChatGPT account” error. More models are available with an API key, but that’s a different approach.

Discovery 3: Gemini CLI Only Supports 2.5 Series

We tested various models with the Gemini CLI (gemini binary).

gemini -p "hello" -m gemini-2.0-flash
# → ModelNotFoundError: models/gemini-2.0-flash is not found

gemini -p "hello" -m gemini-1.5-pro
# → ModelNotFoundError

gemini -p "hello" -m gemini-2.5-flash
# → Works normally

Gemini models accessible with the current account:

gemini-2.5-flash — Default recommended model
gemini-2.5-pro — High-tier
gemini-2.5-flash-lite — Lightweight

Versions of Gemini 2.0 and below return ModelNotFoundError. While this might vary based on account plan or API key type, based on the Gemini CLI, only the 2.5 series worked reliably.

Discovery 4: ZAI Can Be Bypassed with Claude SDK

ZAI is a service that provides an endpoint compatible with the Anthropic API. This allows us to use GLM models with the Claude Code CLI by changing just two environment variables.

ANTHROPIC_BASE_URL=https://<ZAI endpoint> \
ANTHROPIC_AUTH_TOKEN=<ZAI_KEY> \
claude --model glm-5.1 --print "fix the bug"

Since Claude Code internally uses the Anthropic Python SDK, simply overriding ANTHROPIC_BASE_URL allows calling ZAI’s GLM models with the same format. It was interesting that we could reuse the existing claude backend without any separate adapter code.

The three GLM models used were:

glm-5.1 — High-tier
glm-4.7 — Cost-performance balance
glm-4.5-air — Lightweight & High-speed

4-Way Fan-out Comparison Test

We simultaneously issued the same Go bug-fixing task to 4 representative workers out of the 18 (Claude Sonnet, GLM-5.1, Codex gpt-5.5, Gemini 2.5 Flash).

Task: "fix the off-by-one error in the binary search function"

Response times (wall clock):

Worker	Model	Response Time
cc-go-dev-01	claude-sonnet-4-6	~8 seconds
cc-zai-high-dev-01	glm-5.1	~12 seconds
codex-py-dev-01	gpt-5.5	~15 seconds
gemini-py-dev-01	gemini-2.5-flash	~10 seconds

More interesting than the response times were the differences in their approaches. Claude tended to refactor the entire function, while Gemini preferred minimal modifications. Codex often included test code along with the fix.

Of course, this is a single task result and has no statistical significance. It was a verification at the “does it actually work” level, not a benchmark.

Distributed Workers: Adding a Second Host

If all workers are on the same server, the comparative experiment loses some of its meaning. Therefore, we added Claude workers to a second host.

The method for workers to access the NATS broker (on the first host) from the second host is via an autossh tunnel.

[Service]
ExecStart=autossh -N -L 4222:127.0.0.1:4222 broker-host

By forwarding the local port 4222 to the broker, workers can connect to nats://127.0.0.1:4222 from any host without code changes.

Advantage of this method: Workers don’t need to know where the broker is. They can always connect to localhost:4222.

Most Panicked Moment During Operation

The most distressing situation was losing the NATS operator signing key. NATS JetStream uses NKey-based authentication, and the operator/account’s signing key (nsc seed) is required to issue credentials for new workers.

nsc add user --account Services --name new-worker
# → "signing key not found"

There was no backup. Ultimately, we had to perform a large-scale cutover, regenerating the entire NATS operator and replacing all worker credentials with a new permission tree. Service downtime was approximately 60 seconds.

Lesson: Always create an offline backup of the NATS operator seed immediately after generation. If it’s lost, regeneration is the only option.

Summary

Practical conclusions from this experiment:

Claude 3.x is EOL - Inaccessible via Claude Code CLI as of 2026. Use only 4.x.
Codex ChatGPT Account Limited to 4 Models - gpt-5.5, 5.4, 5.4-mini, 5.3-codex. Pro models require a separate API key.
Gemini Only 2.5 Series - Previous versions inaccessible via CLI.
ZAI Integrable via Claude SDK Environment Variable Override - No separate adapter needed.
NATS NKey Must Be Backed Up - Losing the signing key means reissuing everything.

The next installment will cover how these workers are connected, discussing system design and implementation.

AgentForge Blog Automation Service: Full Architecture - From AI Comments to Translation and Post Generation

Tue, 05 May 2026 00:30:00 +0900

Running a blog involves three of the most tedious tasks: replying to comments, maintaining English translations, and consistently writing new posts. The AgentForge project automates all three with AI agents.

This post outlines the complete architecture of our blog automation service, which operates across two servers.

System Topology

┌─────────────────────┐ HTTPS ┌─────────────────────┐
│ arm1 server │ ──────────────▶ │ ec1 server │
│ (Agent Operator) │ │ (Blog Hosting) │
├─────────────────────┤ ├─────────────────────┤
│ blog-agent (:8081) │ │ Hugo (nginx) │
│ ├─ CommentHandler │ │ Blog API (:8000) │
│ ├─ TranslateHandler│ │ ├─ translator.py │
│ └─ PostGenerator │ │ ├─ blog_manager.py │
│ │ │ └─ git_handler.py │
│ NATS / PostgreSQL │ │ │
│ Prometheus / Grafana │ │ Git (yarang/blogs) │
└─────────────────────┘ └─────────────────────┘

Server	Role	Core Services
arm1	Agent Operator	`blog-agent.service` — Flask + Scheduler + LLM Client
ec1	Blog Hosting + API	Hugo (nginx) + `blog-api.service` (FastAPI)

Communication between the two servers is restricted to HTTPS API calls only. SSH access from arm1 to ec1 is blocked, so all integrations are done through the Blog API.

arm1: Unified Blog Agent

Why Unified?

Initially, comment response, translation, and post generation operated as separate processes (three systemd services). The issues were:

Using Claude Code CLI (--print) for calls resulted in a response time of 9.7 seconds and consumed 688MB of disk space.
Managing six systemd units was burdensome.
No state sharing between processes was possible.

By unifying these into one process and switching to direct LLM API calls, we achieved the following:

Metric	Before	After
Response Time	9.7s	1.7s
Disk Usage	688MB	~50MB
systemd Units	6	1
Processes	3	1

Architecture

class BlogAgent:
 """1 Process = Flask (webhook) + Scheduler (timer) + LLM Client"""
 
 def __init__(self):
 self.config = AgentConfig.from_credentials()
 self.llm = LLMClient(self.config) # ZAI glm-4.7
 self.api = BlogAPIClient(self.config) # ec1 Blog API
 
 # Handlers
 self.comment = CommentHandler(self.llm, self.config)
 self.translate = TranslateHandler(self.api)
 self.post_gen = PostGenerator(self.llm, self.api)
 
 # Scheduler
 self.scheduler = Scheduler()
 self.scheduler.every(hours=6, task=self.translate.check_and_sync)
 self.scheduler.daily_at(hour=9, task=self.post_gen.generate_and_publish)

Module Operations

1. CommentHandler — AI Comment Response

Receives Webhook events from GitHub Discussions to automatically generate AI comments.

[User Comment] → GitHub Webhook → arm1 Flask → CommentHandler
 → LLM Call (ZAI glm-4.7) → Generate Reply → Post Comment via GitHub API

Trigger: Webhook event-based (real-time)
Filtering: Skips blog owner comments and AI-generated comments.
Security: HMAC-SHA256 Webhook secret verification, Flask-Limiter applied.

2. TranslateHandler — Automatic Translation Trigger

Requests translation synchronization from ec1’s Blog API every 6 hours.

[Scheduler 6h] → TranslateHandler.check_and_sync()
 → POST /translate/sync → ec1 Blog API performs actual translation

arm1 does not perform the translation itself; it only sends a trigger to the ec1 API. The actual translation logic resides in translator.py on ec1.

3. PostGenerator — Automatic Post Generation

Automatically generates technical blog posts every day at 9 AM.

[Scheduler 09:00 KST] → PostGenerator.generate_and_publish()
 → Collect existing topics → Refer to RSS trends → Generate content with LLM
 → Deduplication Check → Publish via Blog API

Deduplication is key. It compares the similarity between new titles and the last 100 existing titles using difflib.SequenceMatcher:

def _is_duplicate_title(self, new_title, existing_titles):
 """Considers it a duplicate if the ratio is >= 0.6"""
 new_lower = new_title.lower().strip()
 for title in existing_titles[-100:]:
 ex_lower = title.lower().strip()
 ratio = difflib.SequenceMatcher(None, new_lower, ex_lower).ratio()
 if ratio >= 0.6:
 return True
 return False

ec1: Blog API Translation System

Transition to Gemini

Initially, translations were performed using ZAI (glm-4.7), but a critical issue arose:

glm-4.7 is a reasoning model, which first consumes its max_tokens budget for reasoning_content (internal thought process). If max_tokens=256, it uses all 256 tokens for reasoning, leaving the actual content as an empty string.

This led to an incident where nine English posts were translated with empty string titles.

Solution: Replaced with Gemini 2.5 Flash Lite.

Item	ZAI (Previous)	Gemini (Current)
Model	glm-4.7 (reasoning)	gemini-2.5-flash-lite
Translation Time	~30s/post	~8s/post
Cost	Paid API	Free (1,500 requests/day)
Empty Response Issue	Occurred	None

OpenAI-Compatible Endpoint

Gemini provides an OpenAI-compatible API. The existing code can be used without any changes by simply switching the base URL:

LLM_BASE_URLS = {
 "GEMINI": "https://generativelanguage.googleapis.com/v1beta/openai",
 "ZAI": "https://api.z.ai/api/coding/paas/v4",
}

Translation Matching Logic

Pairing Korean↔English posts uses date prefix matching:

ko: 2026-05-04-001-개발-생산성-17배-극대화-deepseek-v4와-...
en: 2026-05-04-001-개발-생산성-17배-극대화-deepseek-v4와-...
 ↑ Same prefix = Same post

Although the slugs might differ in language, if the YYYY-MM-DD-NNN part is the same, it’s recognized as the same post. The prerequisite for this method is that no two posts with the same date and number exist.

Title-in-Body Translation Technique

Translating the title via a separate API call caused issues with empty results from the reasoning model. The solution is to include the title as the first line of the body:

# When requesting translation
prompt = f"# {original_title}\n\n{original_body}"

# Extracting the title from the translation result
if translated.lstrip().startswith("# "):
 lines = translated.lstrip().split("\n", 1)
 extracted_title = lines[0].lstrip("# ").strip()
 translated_body = lines[1].lstrip("\n")

This translates the title and body simultaneously in a single API call, preserving context and saving tokens.

LLM Strategy: Role-Based Model Separation

Not all tasks are handled by a single LLM. Models are separated based on the nature of the task.

Task	Server	Model	Reason
AI Comment Response	arm1	ZAI glm-4.7	Conversational, excellent Korean quality
Post Generation	arm1	ZAI glm-4.7	Long-form content generation, creativity required
Translation (ko→en)	ec1	Gemini Flash Lite	Non-reasoning, fast and free

Core Principle: Do not use reasoning models for translation. Reasoning models consume tokens for internal thought processes, making non-reasoning models more suitable for simple conversion tasks.

Monitoring and Operations

Health Check Endpoints

# arm1 agent
curl http://arm1:8081/health
# → {"status":"healthy","agent":"blog-agent","scheduler_jobs":2,"uptime_sec":...}

curl http://arm1:8081/status
# → {"scheduler":[{"name":"auto-translate","last_run":...},{"name":"post-generator","last_run":"2026-05-04"}]}

# ec1 Blog API
curl https://blog.example.com/api/health
# → {"status":"healthy","version":"2.0.0"}

Observability Points

Metric	Normal Range	Alert Condition
arm1 uptime	>0	Service Down
scheduler_jobs	2	≠ 2
Translation Sync	ko post count = en post count	Discrepancy occurs
Post Generation	1 post daily	No posts for over 24 hours

Lessons Learned and Operational Tips

1. The Pitfall of Reasoning Models

It’s often not explicitly stated in documentation that max_tokens combines reasoning and content. If you get an empty response, check the finish_reason—if it’s "length", it indicates insufficient token budget.

2. Value of the OpenAI-Compatible Pattern

When switching translation providers from ZAI to Gemini, the code change was just one line for the base URL. Abstracting to an OpenAI-compatible interface from the start dramatically reduces LLM replacement costs.

3. Constraints of Date Prefix Matching

In the YYYY-MM-DD-NNN pattern, if two or more posts share the same date and number, translation matching will break. The PostGenerator must include logic to check the last number for that date and increment it when generating new posts.

4. Benefits of Process Consolidation

Consolidating three independent services into one resulted in:

State Sharing (LLM clients, configurations, API clients initialized only once)
Simplified Deployment (one systemd unit)
Easier Debugging (logs consolidated in one place)

Future Plans

Review the integration of arm1 agent’s LLM with Gemini.
Comment Quality Evaluation Pipeline (monitoring the appropriateness of auto-generated comments).
Automatic Translation Quality Verification (comparing with back-translation).
Expanding inter-agent collaboration through the AgentForge framework.

Blog automation aims not for “complete automation,” but for “minimal human intervention.” A structure where AI generates content, humans review it, and the system alerts operators to anomalies is the key to stable operation.

Multi-Model AI Agent Team Design: Composed Architecture and 5-Team Hierarchy

Mon, 30 Mar 2026 00:33:34 +0900

Overview

For building a blog system, I designed a multi-model agent team consisting of 14 AI specialists, 5 teams, and 4 LLM models. The core innovations are two:

Composed Agent: Separating role definitions from execution profiles for maximum reusability
Hierarchical Bridge Leadership: Dual membership of tech leads between upper and lower teams to resolve communication bottlenecks

This post covers the final structure, model distribution strategy, and the composed architecture design process.

Background: Why Multi-Model?

Using a single LLM for all tasks creates two problems:

Cost: Running 14 specialists on a Claude Opus-level model makes costs uncontrollable
Fit: Design needs fast reasoning, security analysis needs deep logic, implementation needs stable coding

So we distributed models based on task characteristics.

Final Team Structure

5 teams, 14 specialists, 4 models.

graph TD
 subgraph UPPER["Upper — Steering Team · consensus"]
 ORC["Orchestrator
relay:steering-orchestrator"]
 DES["Architect
gemini:gemini-2.5-flash"]
 SEC["Security Reviewer
codex:gpt-4o"]
 STL["Backend Tech Lead
relay:developer-zai"]
 FTL["Frontend Tech Lead
relay:developer-zai"]
 DTL["Desktop Tech Lead
relay:developer-zai"]
 INF["Infra Network
gemini:gemini-2.5-flash"]
 SAD["Server Admin
relay:developer-zai"]
 end

 subgraph LOWER_BE["Backend Team · leader_decides"]
 BTL["Backend Tech Lead"]
 BDEV["Backend Developer"]
 end

 subgraph LOWER_FE["Frontend Team · leader_decides"]
 FTL2["Frontend Tech Lead"]
 FDEV["Frontend Developer"]
 FUX["UX Designer"]
 end

 subgraph LOWER_DT["Desktop Team · leader_decides"]
 DTL2["Desktop Tech Lead"]
 DDEV["Desktop Developer"]
 DUX["UX Designer"]
 end

 subgraph LOWER_INFRA["Infra Team · leader_decides"]
 SAD2["Server Admin (Leader)"]
 INET["Cloud Network"]
 DBA["DB Architect"]
 end

 UPPER -.->|bridge| LOWER_BE
 UPPER -.->|bridge| LOWER_FE
 UPPER -.->|bridge| LOWER_DT
 UPPER -.->|bridge| LOWER_INFRA

 BTL --> BDEV
 FTL2 --> FDEV
 FTL2 --> FUX
 DTL2 --> DDEV
 DTL2 --> DUX
 SAD2 --> INET
 SAD2 --> DBA

Team Details

Team	Type	Decision-Making	Leader	Members
Steering Team	upper	consensus	Orchestrator	8 (including bridges)
Backend Team	lower	leader_decides	Backend Tech Lead	2
Frontend Team	lower	leader_decides	Frontend Tech Lead	3
Desktop Team	lower	leader_decides	Desktop Tech Lead	3
Infra Team	lower	leader_decides	Server Admin	3

The Infra Team Separation Decision

In the initial design, DB Architect and Server Admin were part of the backend team. But we separated them based on workspace.

graph LR
 subgraph BackendTeam["Backend Team"]
 B["API Code Writing
FastAPI, Python
workspace: VS Code / SSH"]
 end

 subgraph InfraTeam["Infra Team"]
 S["Server Management
Docker, Ubuntu, Nginx
workspace: SSH Terminal"]
 N["Cloud Network
Cloudflare Dashboard
workspace: Web Console"]
 D["DB Management
PostgreSQL, Migrations
workspace: psql / SSH"]
 end

 B -.->|API deployment| S
 B -.->|Query optimization| D

When workspaces differ, separation is more natural than keeping them together.

Model Distribution Strategy

pie title Specialists by Model
 "relay:developer-zai (GLM)" : 10
 "gemini:gemini-2.5-flash" : 2
 "codex:gpt-4o" : 1
 "zai:glm-4" : 1

Model	Specialists	Purpose	Why Chosen
relay:developer-zai	10	Implementation, ops, leads	Cost-effective, stable coding
gemini:gemini-2.5-flash	2	Design, infra network	Fast response, easy external API calls
codex:gpt-4o	1	Security review	High reasoning, OWASP knowledge
zai:glm-4	1	Context compression	Free tier, text summarization specialized

By assigning 10 implementation specialists to GLM, we achieved 60-70% total cost reduction.

Composed Agent Architecture

The core innovation is separating role definitions (Expert) from execution profiles (Definition).

With the traditional approach, role and execution logic are coupled — any change requires a full rewrite, and reuse is impossible.

Composed Approach

graph TD
 DEF["Definition
Backend Developer"]
 DEF --> BASE["Base: backend-core"]
 DEF --> CAP["Capabilities:
rest-api, crud, auth-jwt"]
 DEF --> PLAT["Platform: fastapi"]
 DEF --> POL["Policy: blog-default"]

 BASE --> |"compose"| RUN["Runtime Agent"]
 CAP --> |"compose"| RUN
 PLAT --> |"compose"| RUN
 POL --> |"compose"| RUN

Module Structure

agent-library/
├── definitions/ ← 14 agent definitions
├── modules/
│ ├── base/ ← 6 base modules
│ ├── capabilities/ ← 15 capability modules
│ ├── platforms/ ← 5 platform modules
│ └── policies/ ← 1 policy
└── runs/ ← execution history

Advantages

Reusability: rest-api capability module shared by backend developer and backend tech lead
Platform swap: Change platform: fastapi to platform: django for instant switching
Capability extension: Add a new capability module and connect it to the Definition
Policy unification: All agents follow the same blog-default policy

Expert-Definition Mapping

Expert	Definition	Base	Capabilities	Platform
Backend Developer	backend-developer	backend-core	rest-api, crud, auth-jwt	fastapi
Backend Tech Lead	backend-tech-lead	backend-core	rest-api, crud, code-review	fastapi
Frontend Developer	frontend-developer	frontend-core	markdown-renderer, list-filter-sort	nextjs
Server Admin	server-administrator	server-core	docker-management, nginx-config, postgres-admin	ubuntu
Infra Network	infra-network-admin	infra-core	dns-management, ssl-certificates, rate-limiting	cloudflare
Security Reviewer	security-auditor	specialist-core	security-audit	fastapi
Context Compressor	context-compressor	specialist-core	context-compression	markdown

TLS Certificate Strategy: Cloudflare Origin CA

For production TLS certificates, we chose Cloudflare Origin CA over Let’s Encrypt.

sequenceDiagram
 participant Client as Visitor
 participant CF as Cloudflare (Proxy)
 participant Nginx as Nginx (Origin)
 participant API as FastAPI

 Client->>CF: HTTPS request
 CF->>CF: Terminate with Cloudflare-managed cert
 CF->>Nginx: Encrypt with Origin CA cert
 Nginx->>API: HTTP (local)
 API-->>Nginx: Response
 Nginx-->>CF: Encrypted with Origin CA
 CF-->>Client: Response

Item	Let’s Encrypt	Cloudflare Origin CA
Validity	90 days (renewal required)	15 years (no renewal)
Issuance	ACME automation required	Manual from Dashboard
Complexity	certbot setup	Copy cert files only

Production architecture:

Oracle Cloud ARM (4 OCPU, 24GB)
├── PostgreSQL (installed directly on host)
├── Docker Compose
│ ├── blog-api (FastAPI)
│ ├── blog-frontend (Next.js standalone)
│ ├── MinIO (S3-compatible storage)
│ └── Nginx (Cloudflare Origin CA)
└── Cloudflare Proxy (Full Strict SSL)

Relay Plugin: Agent Invocation Mechanism

The team structure runs in Claude Code through the Relay plugin.

sequenceDiagram
 participant User as User
 participant Claude as Claude Code
 participant Plugin as Relay Plugin
 participant MCP as MCP Server
 participant LLM as External LLM

 User->>Claude: /relay:invoke-agent
 Claude->>Plugin: Load definition by expert slug
 Plugin->>Plugin: Compose Definition (base + capabilities + platform + policy)
 Plugin->>Plugin: Check backed_by

 alt relay:developer-zai
 Plugin->>Claude: Run internal agent
 else gemini:*
 Plugin->>MCP: Call gemini_mcp server
 MCP->>LLM: Gemini API
 LLM-->>MCP: Response
 MCP-->>Plugin: Result
 else codex:*
 Plugin->>MCP: Call codex_mcp server
 MCP->>LLM: OpenAI API
 LLM-->>MCP: Response
 MCP-->>Plugin: Result
 end

 Plugin-->>Claude: Final result
 Claude-->>User: Response

backed_by Namespaces

Namespace	MCP Server	Purpose
`relay:developer-zai`	internal agent	Implementation, ops (low cost)
`relay:steering-orchestrator`	internal agent	Coordination, final decisions
`gemini:gemini-2.5-flash`	gemini_mcp	Design, external APIs
`codex:gpt-4o`	codex_mcp	Security analysis
`zai:glm-4`	zai_mcp	Context compression

Design Decision Log

Decision	Alternative	Rationale
Separate infra team	Include in backend	Different workspaces (SSH vs IDE)
Cloudflare Origin CA	Let’s Encrypt	15-year validity, no renewal
PostgreSQL on host	Docker container	Memory efficiency on single server
Composed agent	Single-definition agent	Module reusability, easy platform swap
Assign many GLM	Assign many Claude	60-70% cost reduction

Retrospective: What I Learned

1. “Executable Structure” over “Perfect Structure”

Trying to design everything perfectly can prevent you from ever starting. It’s better to compromise and improve as you execute.

2. Workspace Defines Team Boundaries

People who write code and people who manage servers have different physical work environments — that’s a natural team boundary.

3. The Value of Composed Architecture

In an environment with 14 specialists, 5 teams, and 4 models interacting, module separation is essential. It minimizes the scope of changes and maximizes reusability.

4. Cost is Determined at Design Time

Asking “does this task really need a high-cost model?” every time naturally optimizes costs.

Next Steps

Start Phase 1 implementation: DB, Auth, Post/Category CRUD, Docker
Share team operation experience: Problems encountered during actual execution
Performance monitoring: Response time per model, cost vs quality analysis

This post summarizes my experience building an AI agent team using Claude Code + Relay plugin. It reflects learnings from applying this to a real project — different approaches may be more suitable depending on your situation.