AI on Yarang's Tech Lair

Building a Multi-LLM Distributed Orchestrator with NATS JetStream

Fri, 08 May 2026 21:57:11 +0900

Part 1 discussed the model-specific limitations discovered while running four AIs—Claude, ZAI, Codex, and Gemini—concurrently on the same tasks. This part is about “how we made it possible”—the system design and implementation story.

System Overview

AgentForge consists of three components.

[Task Publisher]
 │ NATS JetStream publish
 ▼
[NATS Broker] ─── af.worker.{id}.inbox
 │ JetStream consume (independent streams per worker)
 ▼
[Worker Pollers] × N (poller.py × 18 instances)
 │ LLM CLI Execution (claude / codex / gemini)
 ▼
[Result Return] af.task.{task_id}.completed

When a publisher posts a task to NATS, each worker, which is independently subscribed, receives the message on its inbox and executes the LLM CLI. The result is then published back to a completion topic.

Why NATS JetStream?

We considered several message broker options: Redis Streams, Kafka, RabbitMQ, and NATS JetStream.

Reasons for choosing NATS JetStream:

Single Binary — Operates with a single nats-server without requiring separate runtimes. It has no dependencies like Kafka’s ZooKeeper or RabbitMQ’s Erlang/OTP.
Built-in Persistence — JetStream is a streaming layer on top of NATS, storing messages to the filesystem. This ensures that unprocessed tasks are not lost even if a worker restarts.
NKey-based Authentication — We can issue independent Ed25519 key pairs for each worker. If one worker is compromised, the credentials of other workers remain valid.
Lightweight — Memory usage is around 30MB on a single server. Even with 18 workers connected, the broker load is minimal.

The Core: Backend Adapter in `poller.py`

The heart of the worker is poller.py. This single file handles NATS subscriptions, LLM CLI execution, and result returns.

Since LLMs have different execution methods, we separated them into a backend adapter dictionary.

_BACKENDS: dict[str, dict] = {
 "claude": {
 "bin": os.environ.get("CLAUDE_BIN", "/usr/local/bin/claude"),
 "tools": os.environ.get("ALLOWED_TOOLS", "Read,Edit,Write,Glob,Grep"),
 "model": os.environ.get("CLAUDE_MODEL", ""),
 },
 "codex": {
 "bin": os.environ.get("CODEX_BIN", "/usr/bin/codex"),
 "model": os.environ.get("CODEX_MODEL", ""),
 "sandbox": os.environ.get("CODEX_SANDBOX", "read-only"),
 },
 "gemini_cli": {
 "bin": os.environ.get("GEMINI_BIN", "/usr/bin/gemini"),
 "model": os.environ.get("GEMINI_MODEL", ""),
 },
}

The MODEL_BACKEND environment variable determines which LLM to use. This allows the same poller.py code to run different LLMs across 18 workers.

Claude Backend

async def run_claude(instructions: str, task_id: str) -> tuple[int, str]:
 cfg = _BACKENDS["claude"]
 cmd = [cfg["bin"], "--print", "--allowedTools", cfg["tools"]]
 if cfg.get("model"):
 cmd += ["--model", cfg["model"]]
 proc = await asyncio.create_subprocess_exec(*cmd, stdin=PIPE, stdout=PIPE, stderr=PIPE)

The --print flag is key. It runs Claude Code in non-interactive mode instead of conversational mode, ensuring the results are returned via stdout.

ZAI Backend

ZAI offers an Anthropic API-compatible endpoint, so it doesn’t require a separate backend. Routing is handled by two environment variables.

# /etc/agentforge/cc-zai-high-dev-01.env
ANTHROPIC_BASE_URL=<ZAI endpoint>
ANTHROPIC_AUTH_TOKEN=<ZAI API key>

By injecting this file using systemd’s EnvironmentFile= directive, the claude binary sends requests to the ZAI endpoint. This allows us to connect to a different LLM provider simply by changing environment variables, without altering the code.

Declarative Management: `fleet.yaml` × `servers.yaml`

Manually managing 18 workers is impractical. We declaratively defined the entire infrastructure using two YAML files.

`servers.yaml` — Server Inventory

servers:
 - name: worker-node-1
 role: worker-host
 services: [agentforge-worker, tunnel-arm1]

 - name: broker-host
 role: broker-host
 services: [nats-jetstream, postgres]

 - name: worker-node-2
 role: worker-host
 services: [agentforge-worker, tunnel-arm1]

`fleet.yaml` — Worker Placement

workers:
 - worker_id: cc-go-dev-01
 llm: claude-code
 model: claude-sonnet-4-6
 lang: go
 role: developer
 host: worker-node-1
 enabled: true
 create_pr: true

 - worker_id: codex-py-dev-01
 llm: codex
 model: gpt-5.5
 lang: python
 role: developer
 host: worker-node-1
 enabled: true
 create_pr: false

Changing just the host field moves a worker to a different server. Setting enabled: false stops the deployment script from starting that worker.

Worker Templating System: `provision_worker.py`

Manually writing systemd unit files for each new worker is prone to errors. We automated this using Jinja2 templates and a provisioning script.

Template Structure

templates/
 systemd/
 claude.service.j2 # For claude-code and ZAI alike
 codex.service.j2 # OpenAI Codex
 gemini.service.j2 # Google Gemini CLI

The core part of claude.service.j2:

Environment=MODEL_BACKEND=claude
Environment=CLAUDE_BIN={{ claude_bin }}
{% if claude_model %}
Environment=CLAUDE_MODEL={{ claude_model }}
{% endif %}
{% if env_file %}
EnvironmentFile={{ env_file }}
{% endif %}
Environment=WORK_BASE={{ work_base }}
Environment=WORK_DIR={{ work_base }}/repo
Environment="{{ 'ALLOWED_TOOLS=' + allowed_tools }}"
Environment=CREATE_PR={{ 'true' if create_pr else 'false' }}
{% if create_pr and github_remote %}
Environment=GITHUB_REMOTE={{ github_remote }}
{% endif %}

For ZAI workers, the env_file block is activated, adding the EnvironmentFile. For PR creation workers, github_remote is injected. Other settings use defaults.

`provision_worker.py` Usage

# Preview (no actual deployment)
python3 scripts/provision_worker.py --worker new-worker-id --dry-run

# Actual deployment (including NATS creds issuance)
python3 scripts/provision_worker.py --worker new-worker-id --issue-creds

# Bulk deployment for the entire fleet.yaml
python3 scripts/provision_worker.py --all

Internal operations:

Reads worker entries from fleet.yaml.
Reads target hosts from servers.yaml.
Renders Jinja2 templates.
Deploys /etc/systemd/system/{worker_id}-poller.service via SSH.
Creates the working directory.
Executes systemctl daemon-reload && enable --now.
(Optional) Issues NATS NKey with nsc add user → deploys creds → regenerates auth.conf.

Distributed Hosting: Adding Workers to a Second Server

Running all workers on a single server creates a single point of failure. We added Claude workers to a second host.

The method for workers on the second host to connect to the NATS broker is via an autossh tunnel.

[Unit]
Description=NATS Broker Tunnel
After=network-online.target

[Service]
ExecStart=/usr/bin/autossh -N \
 -L 4222:127.0.0.1:4222 \
 -i /home/ubuntu/.ssh/id_ed25519 \
 broker-host
Restart=always
RestartSec=10

With this configuration active, workers always connect to nats://127.0.0.1:4222. They don’t need to know the broker host’s address. As long as the tunnel is alive, it works the same way from any host.

NATS Credential Operations Experience

NATS NKey management was the most complex part of the implementation.

NATS JetStream’s authentication structure is hierarchical.

Operator (Root Signing Authority)
 └── Account: SYS (System Account)
 └── Account: Services (Worker Account)
 ├── User: cc-dev-01
 ├── User: cc-go-dev-01
 ├── User: codex-py-dev-01
 └── ...

Each worker has an independent User NKey and can publish/subscribe within the permissions scope (af.>, _INBOX.>, $JS.>) of the Services account.

Adding a new worker requires the Operator’s signing key. We initially made the mistake of not backing up this key, leading to its loss. Consequently, we had to regenerate the entire Operator and replace all worker credentials en masse. The service downtime was approximately 60 seconds.

# Regeneration procedure
nsc add operator AgentForge
nsc add account SYS
nsc add account Services
for worker in cc-dev-01 cc-go-dev-01 ...; do
 nsc add user --account Services --name $worker \
 --allow-pub "af.>,_INBOX.>,$JS.>" \
 --allow-sub "af.>,_INBOX.>,$JS.>"
done
nsc generate config --mem-resolver --sys-account SYS > auth.new.conf

Adding a New Worker: The Full Procedure

Since the completion of this system, adding a new worker is straightforward.

Step 1: Add an entry to fleet.yaml

- worker_id: my-new-worker
 llm: claude-code
 model: claude-haiku-4-5
 lang: multi
 role: developer
 host: worker-node-1
 enabled: true
 create_pr: false

Step 2: Preview

python3 scripts/provision_worker.py --worker my-new-worker --dry-run

Step 3: Actual Deployment

python3 scripts/provision_worker.py --worker my-new-worker --issue-creds

That’s it. Template rendering, SSH deployment, NATS credential issuance, and service registration are all handled by a single command.

Next Steps

The current system is structured such that workers process tasks independently. Future plans include:

Routing Policies: Automatically selecting the appropriate worker based on task characteristics (e.g., Go code → claude-go-dev, cost-first → ZAI lightweight tier).
Results Comparison Dashboard: A UI to display fan-out results side-by-side.
Cost Tracking: Aggregating API call costs per worker.

The code is publicly available on GitHub.

I Sent the Same Coding Task to 4 AIs Simultaneously

Fri, 08 May 2026 21:55:39 +0900

What happens when the same bug-fixing task is sent to Claude, ZAI (GLM), OpenAI Codex, and Google Gemini simultaneously?

This question sparked the AgentForge project. We built a system that connects multiple LLM CLIs with the NATS JetStream message queue to process the same tasks in parallel, and in the process, we made some unexpected discoveries. This article focuses on the comparative experimental findings during the setup phase.

The system’s design and implementation will be covered in Part 2.

List of AIs Tested

The final configuration of 18 operational workers is as follows:

Family	Model	Notes
Claude Code	claude-sonnet-4-6	Main development worker
Claude Code	claude-sonnet-4-5	Previous generation comparison
Claude Code	claude-haiku-4-5	Lightweight & High-speed
Claude Code	claude-opus-4-6	Top-tier
Claude Code	claude-opus-4-5	Previous generation comparison
ZAI (GLM)	glm-5.1	High-tier
ZAI (GLM)	glm-4.7	Mid-tier
ZAI (GLM)	glm-4.5-air	Lightweight tier
OpenAI Codex	gpt-5.5
Codex	gpt-5.4	1M context
Codex	gpt-5.4-mini	400K context
Codex	gpt-5.3-codex	272K context
Google Gemini	gemini-2.5-flash
Gemini	gemini-2.5-pro	High-tier
Gemini	gemini-2.5-flash-lite	Lightweight

The list was much shorter when we first started. It grew as we experimented with which models were available.

Discovery 1: Claude 3.x Series is Already Inaccessible

Those who have used Claude Code for a long time might recall Claude 3.7 Sonnet, 3.5 Sonnet, and 3.5 Haiku. We attempted to add these models as workers.

claude --model claude-3-7-sonnet-20250219 --print "hello"
# → "may not exist or no access"

All three models returned the same error. The Claude 3 series reached its EOL in early 2026, and access via the Claude Code CLI has been blocked. Currently, only the 4.x series is available with a Claude Code subscription.

Conclusion: Claude workers were configured using only the 4.5/4.6 series.

Discovery 2: Limited Model Selection for ChatGPT Account Codex

The OpenAI Codex CLI authenticates with a ChatGPT Plus/Pro account or a separate API key. If authenticated via a ChatGPT account, the accessible models are limited.

codex --model gpt-5.5-pro "fix the bug"
# → "Model gpt-5.5-pro is not supported with ChatGPT account"

codex --model gpt-5.5 "fix the bug"
# → Works normally

Models available with a ChatGPT account:

Model	Context	Inference Level
gpt-5.5	1M / 1M	High
gpt-5.4	1M / 1M	Medium
gpt-5.4-mini	400K / 400K	Medium
gpt-5.3-codex	272K / 400K	Medium

All other models, including gpt-5.5-pro, returned a “not supported with ChatGPT account” error. More models are available with an API key, but that’s a different approach.

Discovery 3: Gemini CLI Only Supports 2.5 Series

We tested various models with the Gemini CLI (gemini binary).

gemini -p "hello" -m gemini-2.0-flash
# → ModelNotFoundError: models/gemini-2.0-flash is not found

gemini -p "hello" -m gemini-1.5-pro
# → ModelNotFoundError

gemini -p "hello" -m gemini-2.5-flash
# → Works normally

Gemini models accessible with the current account:

gemini-2.5-flash — Default recommended model
gemini-2.5-pro — High-tier
gemini-2.5-flash-lite — Lightweight

Versions of Gemini 2.0 and below return ModelNotFoundError. While this might vary based on account plan or API key type, based on the Gemini CLI, only the 2.5 series worked reliably.

Discovery 4: ZAI Can Be Bypassed with Claude SDK

ZAI is a service that provides an endpoint compatible with the Anthropic API. This allows us to use GLM models with the Claude Code CLI by changing just two environment variables.

ANTHROPIC_BASE_URL=https://<ZAI endpoint> \
ANTHROPIC_AUTH_TOKEN=<ZAI_KEY> \
claude --model glm-5.1 --print "fix the bug"

Since Claude Code internally uses the Anthropic Python SDK, simply overriding ANTHROPIC_BASE_URL allows calling ZAI’s GLM models with the same format. It was interesting that we could reuse the existing claude backend without any separate adapter code.

The three GLM models used were:

glm-5.1 — High-tier
glm-4.7 — Cost-performance balance
glm-4.5-air — Lightweight & High-speed

4-Way Fan-out Comparison Test

We simultaneously issued the same Go bug-fixing task to 4 representative workers out of the 18 (Claude Sonnet, GLM-5.1, Codex gpt-5.5, Gemini 2.5 Flash).

Task: "fix the off-by-one error in the binary search function"

Response times (wall clock):

Worker	Model	Response Time
cc-go-dev-01	claude-sonnet-4-6	~8 seconds
cc-zai-high-dev-01	glm-5.1	~12 seconds
codex-py-dev-01	gpt-5.5	~15 seconds
gemini-py-dev-01	gemini-2.5-flash	~10 seconds

More interesting than the response times were the differences in their approaches. Claude tended to refactor the entire function, while Gemini preferred minimal modifications. Codex often included test code along with the fix.

Of course, this is a single task result and has no statistical significance. It was a verification at the “does it actually work” level, not a benchmark.

Distributed Workers: Adding a Second Host

If all workers are on the same server, the comparative experiment loses some of its meaning. Therefore, we added Claude workers to a second host.

The method for workers to access the NATS broker (on the first host) from the second host is via an autossh tunnel.

[Service]
ExecStart=autossh -N -L 4222:127.0.0.1:4222 broker-host

By forwarding the local port 4222 to the broker, workers can connect to nats://127.0.0.1:4222 from any host without code changes.

Advantage of this method: Workers don’t need to know where the broker is. They can always connect to localhost:4222.

Most Panicked Moment During Operation

The most distressing situation was losing the NATS operator signing key. NATS JetStream uses NKey-based authentication, and the operator/account’s signing key (nsc seed) is required to issue credentials for new workers.

nsc add user --account Services --name new-worker
# → "signing key not found"

There was no backup. Ultimately, we had to perform a large-scale cutover, regenerating the entire NATS operator and replacing all worker credentials with a new permission tree. Service downtime was approximately 60 seconds.

Lesson: Always create an offline backup of the NATS operator seed immediately after generation. If it’s lost, regeneration is the only option.

Summary

Practical conclusions from this experiment:

Claude 3.x is EOL - Inaccessible via Claude Code CLI as of 2026. Use only 4.x.
Codex ChatGPT Account Limited to 4 Models - gpt-5.5, 5.4, 5.4-mini, 5.3-codex. Pro models require a separate API key.
Gemini Only 2.5 Series - Previous versions inaccessible via CLI.
ZAI Integrable via Claude SDK Environment Variable Override - No separate adapter needed.
NATS NKey Must Be Backed Up - Losing the signing key means reissuing everything.

The next installment will cover how these workers are connected, discussing system design and implementation.

Building a Blog AI Auto-Comment System (1/3) — Architecture and Implementation

Sun, 03 May 2026 01:00:00 +0900

Overview

I built a system where AI automatically responds to blog comments. When a reader leaves a comment on a blog post, the AI assistant analyzes the post context and automatically generates a technically accurate yet friendly reply.

This series consists of three parts:

Part 1 (This post): Overall architecture design and core code implementation
Part 2: File-based authentication, permission management, security hardening
Part 3: systemd deployment, nginx proxy, troubleshooting

System Architecture

Overall Data Flow

Reader writes a comment
 ↓
[giscus] → Creates a comment on GitHub Discussions
 ↓
[GitHub Webhook] → Sends HTTP POST
 ↓
[nginx reverse proxy] → Header forwarding
 ↓
[Flask Worker] → Signature verification → Comment analysis
 ↓
[Claude Code CLI] → Generates AI response
 ↓
[GitHub GraphQL API] → Posts reply to Discussion
 ↓
[giscus] → Displays reply on blog

The key point in this architecture is that giscus uses GitHub Discussions as a comment storage. Therefore, we can receive new comment events via GitHub Webhook and post responses using the same GitHub API.

Components

Component	Role	Technology
giscus	Blog comment widget	Based on GitHub Discussions
GitHub Webhook	Event delivery	`discussion_comment` event
nginx	Reverse proxy	Header forwarding, SSL termination
Flask Worker	Webhook receiving and processing	Python, Flask, Flask-Limiter
Claude Code	AI response generation	`--print` mode CLI call
GitHub GraphQL	Response posting	Mutation API

giscus Configuration

To integrate giscus into a Hugo blog, the following configuration is required in hugo.toml:

[params]
 [params.comments]
 enabled = true
 provider = "giscus"
 [params.comments.giscus]
 repo = "yarang/blogs"
 repoId = "YOUR_REPO_ID"
 category = "General"
 categoryId = "YOUR_CATEGORY_ID"
 mapping = "pathname"
 strict = "0"
 reactionsEnabled = "1"
 emitMetadata = "0"
 inputPosition = "bottom"
 lang = "ko"
 theme = "noborder_gray"

mapping = "pathname" maps Discussions based on the post URL path. This creates an independent Discussion for each blog post.

GitHub Webhook Configuration

In the GitHub repository Settings > Webhooks:

Payload URL: https://your-domain/webhook
Content type: application/json
Secret: Secret for HMAC-SHA256 signature (covered in detail in the security post)
Events: Select Discussion comments

The Webhook sends a discussion_comment event to the Flask worker whenever a comment is created.

Flask Worker Implementation

Project Structure

auto-comment-worker/
├── scripts/
│ └── auto-comment-worker.py # Main worker
├── deploy/
│ └── auto-comment-worker.service # systemd service
├── venv/ # Python virtual environment
└── logs/
 └── audit.log # Audit log

Core Dependencies

from flask import Flask, request, jsonify
from flask_limiter import Limiter
from flask_limiter.util import get_remote_address
from marshmallow import Schema, fields, validate, ValidationError

Flask: Provides Webhook endpoints as a lightweight web framework
Flask-Limiter: Prevents abuse with rate limiting (10 times per minute)
marshmallow: Safe data parsing via request schema validation

Webhook Endpoint

@app.route('/webhook', methods=['POST'])
@limiter.limit("10 per minute")
def github_webhook():
 """Receives GitHub Webhook"""
 # 1. Signature verification
 signature = request.headers.get('X-Hub-Signature-256')
 if not verify_webhook_signature(request.data, signature):
 log_audit('SIGNATURE_INVALID', {'ip': request.remote_addr})
 return jsonify({'status': 'unauthorized'}), 401

 # 2. Request schema validation
 schema = WebhookSchema()
 try:
 payload = schema.load(request.json)
 except ValidationError as err:
 return jsonify({'status': 'invalid'}), 400

 # 3. Extract and filter comment information
 comment = payload.get('comment', {})
 discussion = payload.get('discussion', {})
 original_author = comment.get('user', {}).get('login', 'User')

 # Ignore if it's the blog owner's comment
 if _is_blog_owner(original_author):
 return jsonify({'status': 'owner_comment_ignored'}), 200

 # Ignore if AI generated comment (prevent infinite loop)
 if _is_ai_generated_comment(comment.get('body', '')):
 return jsonify({'status': 'ai_comment_ignored'}), 200

 # 4. Generate and post AI response
 discussion_graphql_id = get_discussion_graphql_id(
 repo_owner, repo_name, discussion_number
 )
 context = f"Title: {discussion_title}\n\nContent: {discussion_body}"
 reply = analyze_comment(context, comment_body)
 post_reply_graphql(discussion_graphql_id, comment_body, original_author, reply)

The core flow consists of 4 steps:

Signature verification: Verifies if the request came from GitHub using HMAC-SHA256
Schema validation: Validates payload structure with marshmallow
Filtering: Ignores comments from the blog owner and the AI itself (prevents infinite loops)
Response: Generates AI response with Claude Code and posts via GraphQL API

Preventing Infinite Loops

If AI responds to a comment created by AI, it will fall into an infinite loop. To prevent this, we use marker-based detection:

def _is_ai_generated_comment(body: str) -> bool:
 """Identifies if the comment was generated by AI"""
 ai_markers = [
 '🤖 AI Assistant',
 'AI Assistant',
 'AgentForge',
 'Automatically generated by Claude Code',
 'was automatically generated'
 ]
 body_lower = body.lower()
 return any(marker.lower() in body_lower for marker in ai_markers)

When posting the AI response, you must include one of these markers in the body:

body = f"""---
**🤖 AI Assistant**

{safe_reply}

*This comment was automatically generated by AgentForge + Claude Code.*
---
"""

Calling Claude Code CLI

Use Claude Code’s --print mode to generate an AI response non-interactively:

def analyze_comment(context: str, comment: str) -> str:
 """Run Claude Code with AgentForge settings"""
 prompt = f"""## Blog Post Context
{context[:2000]}

## Reader Comment
{comment}

Please write a response to this comment.
- As a technical blog assistant, be professional yet friendly
- Concise within 200 characters
- Provide additional info or links if needed
"""

 cmd = [
 CLAUDE_CODE_PATH,
 '--settings', AGENTFORGE_CONFIG,
 '--print', prompt
 ]

 result = run(cmd, capture_output=True, text=True, timeout=60)

 if result.returncode == 0 and result.stdout.strip():
 return result.stdout.strip()

 return "Thanks for your opinion! I think it would be great to discuss the technical aspects further. 🙏"

Specify the AgentForge dedicated configuration file with the --settings flag. You can manage the model, token limits, etc. in this configuration file.

The --print flag runs Claude Code in non-interactive mode and outputs the result to stdout. Unlike interactive mode, it terminates after a single prompt-response.

GitHub GraphQL API Integration

Since giscus uses GitHub Discussions, the response must also be posted via the GitHub GraphQL API.

Retrieve Discussion GraphQL ID:

def get_discussion_graphql_id(repo_owner, repo_name, discussion_number):
 query = """
 query($owner: String!, $name: String!, $number: Int!) {
 repository(owner: $owner, name: $name) {
 discussion(number: $number) {
 id
 }
 }
 }
 """
 variables = {
 "owner": repo_owner,
 "name": repo_name,
 "number": discussion_number
 }

 response = requests.post(
 GITHUB_API_URL,
 json={"query": query, "variables": variables},
 headers={
 "Authorization": f"Bearer {GITHUB_TOKEN}",
 "Content-Type": "application/json"
 },
 timeout=10
 )

 if response.status_code == 200:
 data = response.json()
 return data["data"]["repository"]["discussion"]["id"]
 return None

The Webhook payload only includes the Discussion’s REST API ID. However, to post a comment, the GraphQL Node ID is required. Therefore, we first retrieve the Discussion’s Node ID via a GraphQL query, then use it to post the comment.

Post Response (Mutation):

def post_reply_graphql(discussion_graphql_id, original_comment, original_author, reply):
 query = """
 mutation($discussionId: ID!, $body: String!) {
 addDiscussionComment(input: {
 discussionId: $discussionId, body: $body
 }) {
 comment { id, databaseId }
 }
 }
 """
 # ... (Send request)

The addDiscussionComment mutation adds a new comment to the entire Discussion. It is a Discussion-level comment, not a reply to a specific comment.

Input Sanitization

Since user comments are external input, they must be sanitized:

def sanitize_comment(body: str) -> str:
 """User input sanitization"""
 if not body:
 return body
 body = re.sub(r'<[^>]+>', '', body) # Remove HTML tags
 body = html.escape(body) # Escape HTML entities
 body = body[:1000] # Length limit
 return body

To prevent XSS attacks, HTML tags are removed, remaining special characters are escaped, and the length is limited to 1000 characters.

Audit Logging

We keep audit logs to track security events:

def log_audit(event_type: str, details: dict):
 """Record security event audit log"""
 with open(AUDIT_LOG, 'a') as f:
 f.write(json.dumps({
 'timestamp': datetime.utcnow().isoformat(),
 'event': event_type,
 'details': details
 }) + '\n')

Event types recorded:

SIGNATURE_INVALID: Webhook signature verification failed
INVALID_PAYLOAD: Invalid request payload
WEBHOOK_RECEIVED: Webhook received successfully
AI_RESPONSE_SENT: AI response posted successfully

Conclusion

In this Part 1, we covered the overall architecture and core implementation code connecting giscus, GitHub Webhook, Flask worker, Claude Code CLI, and GitHub GraphQL API.

In the upcoming Part 2, we will cover security hardening for this system — file-based authentication management, file permission verification, HMAC-SHA256 signature verification, and more in detail.

This post is Part 1 of the AgentForge blog automatic comment system series.

Multi-Model AI Agent Team Design: Composed Architecture and 5-Team Hierarchy

Mon, 30 Mar 2026 00:33:34 +0900

Overview

For building a blog system, I designed a multi-model agent team consisting of 14 AI specialists, 5 teams, and 4 LLM models. The core innovations are two:

Composed Agent: Separating role definitions from execution profiles for maximum reusability
Hierarchical Bridge Leadership: Dual membership of tech leads between upper and lower teams to resolve communication bottlenecks

This post covers the final structure, model distribution strategy, and the composed architecture design process.

Background: Why Multi-Model?

Using a single LLM for all tasks creates two problems:

Cost: Running 14 specialists on a Claude Opus-level model makes costs uncontrollable
Fit: Design needs fast reasoning, security analysis needs deep logic, implementation needs stable coding

So we distributed models based on task characteristics.

Final Team Structure

5 teams, 14 specialists, 4 models.

graph TD
 subgraph UPPER["Upper — Steering Team · consensus"]
 ORC["Orchestrator
relay:steering-orchestrator"]
 DES["Architect
gemini:gemini-2.5-flash"]
 SEC["Security Reviewer
codex:gpt-4o"]
 STL["Backend Tech Lead
relay:developer-zai"]
 FTL["Frontend Tech Lead
relay:developer-zai"]
 DTL["Desktop Tech Lead
relay:developer-zai"]
 INF["Infra Network
gemini:gemini-2.5-flash"]
 SAD["Server Admin
relay:developer-zai"]
 end

 subgraph LOWER_BE["Backend Team · leader_decides"]
 BTL["Backend Tech Lead"]
 BDEV["Backend Developer"]
 end

 subgraph LOWER_FE["Frontend Team · leader_decides"]
 FTL2["Frontend Tech Lead"]
 FDEV["Frontend Developer"]
 FUX["UX Designer"]
 end

 subgraph LOWER_DT["Desktop Team · leader_decides"]
 DTL2["Desktop Tech Lead"]
 DDEV["Desktop Developer"]
 DUX["UX Designer"]
 end

 subgraph LOWER_INFRA["Infra Team · leader_decides"]
 SAD2["Server Admin (Leader)"]
 INET["Cloud Network"]
 DBA["DB Architect"]
 end

 UPPER -.->|bridge| LOWER_BE
 UPPER -.->|bridge| LOWER_FE
 UPPER -.->|bridge| LOWER_DT
 UPPER -.->|bridge| LOWER_INFRA

 BTL --> BDEV
 FTL2 --> FDEV
 FTL2 --> FUX
 DTL2 --> DDEV
 DTL2 --> DUX
 SAD2 --> INET
 SAD2 --> DBA

Team Details

Team	Type	Decision-Making	Leader	Members
Steering Team	upper	consensus	Orchestrator	8 (including bridges)
Backend Team	lower	leader_decides	Backend Tech Lead	2
Frontend Team	lower	leader_decides	Frontend Tech Lead	3
Desktop Team	lower	leader_decides	Desktop Tech Lead	3
Infra Team	lower	leader_decides	Server Admin	3

The Infra Team Separation Decision

In the initial design, DB Architect and Server Admin were part of the backend team. But we separated them based on workspace.

graph LR
 subgraph BackendTeam["Backend Team"]
 B["API Code Writing
FastAPI, Python
workspace: VS Code / SSH"]
 end

 subgraph InfraTeam["Infra Team"]
 S["Server Management
Docker, Ubuntu, Nginx
workspace: SSH Terminal"]
 N["Cloud Network
Cloudflare Dashboard
workspace: Web Console"]
 D["DB Management
PostgreSQL, Migrations
workspace: psql / SSH"]
 end

 B -.->|API deployment| S
 B -.->|Query optimization| D

When workspaces differ, separation is more natural than keeping them together.

Model Distribution Strategy

pie title Specialists by Model
 "relay:developer-zai (GLM)" : 10
 "gemini:gemini-2.5-flash" : 2
 "codex:gpt-4o" : 1
 "zai:glm-4" : 1

Model	Specialists	Purpose	Why Chosen
relay:developer-zai	10	Implementation, ops, leads	Cost-effective, stable coding
gemini:gemini-2.5-flash	2	Design, infra network	Fast response, easy external API calls
codex:gpt-4o	1	Security review	High reasoning, OWASP knowledge
zai:glm-4	1	Context compression	Free tier, text summarization specialized

By assigning 10 implementation specialists to GLM, we achieved 60-70% total cost reduction.

Composed Agent Architecture

The core innovation is separating role definitions (Expert) from execution profiles (Definition).

With the traditional approach, role and execution logic are coupled — any change requires a full rewrite, and reuse is impossible.

Composed Approach

graph TD
 DEF["Definition
Backend Developer"]
 DEF --> BASE["Base: backend-core"]
 DEF --> CAP["Capabilities:
rest-api, crud, auth-jwt"]
 DEF --> PLAT["Platform: fastapi"]
 DEF --> POL["Policy: blog-default"]

 BASE --> |"compose"| RUN["Runtime Agent"]
 CAP --> |"compose"| RUN
 PLAT --> |"compose"| RUN
 POL --> |"compose"| RUN

Module Structure

agent-library/
├── definitions/ ← 14 agent definitions
├── modules/
│ ├── base/ ← 6 base modules
│ ├── capabilities/ ← 15 capability modules
│ ├── platforms/ ← 5 platform modules
│ └── policies/ ← 1 policy
└── runs/ ← execution history

Advantages

Reusability: rest-api capability module shared by backend developer and backend tech lead
Platform swap: Change platform: fastapi to platform: django for instant switching
Capability extension: Add a new capability module and connect it to the Definition
Policy unification: All agents follow the same blog-default policy

Expert-Definition Mapping

Expert	Definition	Base	Capabilities	Platform
Backend Developer	backend-developer	backend-core	rest-api, crud, auth-jwt	fastapi
Backend Tech Lead	backend-tech-lead	backend-core	rest-api, crud, code-review	fastapi
Frontend Developer	frontend-developer	frontend-core	markdown-renderer, list-filter-sort	nextjs
Server Admin	server-administrator	server-core	docker-management, nginx-config, postgres-admin	ubuntu
Infra Network	infra-network-admin	infra-core	dns-management, ssl-certificates, rate-limiting	cloudflare
Security Reviewer	security-auditor	specialist-core	security-audit	fastapi
Context Compressor	context-compressor	specialist-core	context-compression	markdown

TLS Certificate Strategy: Cloudflare Origin CA

For production TLS certificates, we chose Cloudflare Origin CA over Let’s Encrypt.

sequenceDiagram
 participant Client as Visitor
 participant CF as Cloudflare (Proxy)
 participant Nginx as Nginx (Origin)
 participant API as FastAPI

 Client->>CF: HTTPS request
 CF->>CF: Terminate with Cloudflare-managed cert
 CF->>Nginx: Encrypt with Origin CA cert
 Nginx->>API: HTTP (local)
 API-->>Nginx: Response
 Nginx-->>CF: Encrypted with Origin CA
 CF-->>Client: Response

Item	Let’s Encrypt	Cloudflare Origin CA
Validity	90 days (renewal required)	15 years (no renewal)
Issuance	ACME automation required	Manual from Dashboard
Complexity	certbot setup	Copy cert files only

Production architecture:

Oracle Cloud ARM (4 OCPU, 24GB)
├── PostgreSQL (installed directly on host)
├── Docker Compose
│ ├── blog-api (FastAPI)
│ ├── blog-frontend (Next.js standalone)
│ ├── MinIO (S3-compatible storage)
│ └── Nginx (Cloudflare Origin CA)
└── Cloudflare Proxy (Full Strict SSL)

Relay Plugin: Agent Invocation Mechanism

The team structure runs in Claude Code through the Relay plugin.

sequenceDiagram
 participant User as User
 participant Claude as Claude Code
 participant Plugin as Relay Plugin
 participant MCP as MCP Server
 participant LLM as External LLM

 User->>Claude: /relay:invoke-agent
 Claude->>Plugin: Load definition by expert slug
 Plugin->>Plugin: Compose Definition (base + capabilities + platform + policy)
 Plugin->>Plugin: Check backed_by

 alt relay:developer-zai
 Plugin->>Claude: Run internal agent
 else gemini:*
 Plugin->>MCP: Call gemini_mcp server
 MCP->>LLM: Gemini API
 LLM-->>MCP: Response
 MCP-->>Plugin: Result
 else codex:*
 Plugin->>MCP: Call codex_mcp server
 MCP->>LLM: OpenAI API
 LLM-->>MCP: Response
 MCP-->>Plugin: Result
 end

 Plugin-->>Claude: Final result
 Claude-->>User: Response

backed_by Namespaces

Namespace	MCP Server	Purpose
`relay:developer-zai`	internal agent	Implementation, ops (low cost)
`relay:steering-orchestrator`	internal agent	Coordination, final decisions
`gemini:gemini-2.5-flash`	gemini_mcp	Design, external APIs
`codex:gpt-4o`	codex_mcp	Security analysis
`zai:glm-4`	zai_mcp	Context compression

Design Decision Log

Decision	Alternative	Rationale
Separate infra team	Include in backend	Different workspaces (SSH vs IDE)
Cloudflare Origin CA	Let’s Encrypt	15-year validity, no renewal
PostgreSQL on host	Docker container	Memory efficiency on single server
Composed agent	Single-definition agent	Module reusability, easy platform swap
Assign many GLM	Assign many Claude	60-70% cost reduction

Retrospective: What I Learned

1. “Executable Structure” over “Perfect Structure”

Trying to design everything perfectly can prevent you from ever starting. It’s better to compromise and improve as you execute.

2. Workspace Defines Team Boundaries

People who write code and people who manage servers have different physical work environments — that’s a natural team boundary.

3. The Value of Composed Architecture

In an environment with 14 specialists, 5 teams, and 4 models interacting, module separation is essential. It minimizes the scope of changes and maximizes reusability.

4. Cost is Determined at Design Time

Asking “does this task really need a high-cost model?” every time naturally optimizes costs.

Next Steps

Start Phase 1 implementation: DB, Auth, Post/Category CRUD, Docker
Share team operation experience: Problems encountered during actual execution
Performance monitoring: Response time per model, cost vs quality analysis

This post summarizes my experience building an AI agent team using Claude Code + Relay plugin. It reflects learnings from applying this to a real project — different approaches may be more suitable depending on your situation.

[LLM] How to Write Effective Prompts

Sat, 21 Feb 2026 20:40:00 +0900

Introduction

Writing effective prompts is essential for getting the most out of LLMs (Large Language Models). This article summarizes key principles and practical patterns of prompt engineering.

Core Principles of Good Prompts

1. Clarity

Avoid ambiguous expressions and be specific.

Bad Example:

Write good code

Good Example:

Implement a binary search tree in Python.
Include insert, search, and delete methods,
with time complexity of O(log n).

2. Provide Context

Give background information needed for the LLM to understand the task.

I'm a React beginner.
Explain the difference between useState and useEffect
with example code.

3. Role Playing

Set up responses from a specific expert’s perspective.

You are a senior backend developer with 10 years of experience.
Explain the pros and cons of microservices architecture.

4. Specify Output Format

Explicitly state the desired response format.

Summarize the following in a markdown table:
- Language features
- Pros and cons
- Use cases

Prompt Patterns

Chain of Thought

Guide step-by-step thinking for complex problems.

Let's think through this problem step by step:
1. First analyze the problem
2. Consider solutions
3. Write the final answer

Few-Shot Learning

Provide examples to teach the desired format.

Summarize in the following format:

Input: "The weather is nice today"
Output: Positive, Weather

Input: "The meeting was too long"
Output: Negative, Work

Input: "Started a new project"
Output: ?

Structured Prompts

Divide complex tasks into sections.

## Goal
Design a user authentication API

## Requirements
- Use JWT tokens
- Refresh token rotation
- Apply rate limiting

## Output
1. API endpoint specification
2. Sequence diagram
3. Security considerations

Common Mistakes

Mistake	Problem	Solution
Too long prompts	Key points get lost	Keep it concise
Ambiguous instructions	Unexpected results	Provide specific examples
Missing context	Inaccurate answers	Add background info
Unspecified format	Poor readability	Specify output format

Practical Checklist

Check before writing your prompt:

Is the goal clear?
Included necessary context?
Specified output format?
Stated constraints?
Would examples help?

Conclusion

Good prompts are clear, specific, and provide necessary context. Practice to improve your prompt writing skills.

References

Korean Version: 한국어 버전

AI on Yarang's Tech Lair

Building a Multi-LLM Distributed Orchestrator with NATS JetStream

System Overview

Why NATS JetStream?

The Core: Backend Adapter in poller.py

Claude Backend

ZAI Backend

Declarative Management: fleet.yaml × servers.yaml

servers.yaml — Server Inventory

fleet.yaml — Worker Placement

Worker Templating System: provision_worker.py

Template Structure

provision_worker.py Usage

Distributed Hosting: Adding Workers to a Second Server

NATS Credential Operations Experience

Adding a New Worker: The Full Procedure

Next Steps

I Sent the Same Coding Task to 4 AIs Simultaneously

List of AIs Tested

Discovery 1: Claude 3.x Series is Already Inaccessible

Discovery 2: Limited Model Selection for ChatGPT Account Codex

Discovery 3: Gemini CLI Only Supports 2.5 Series

Discovery 4: ZAI Can Be Bypassed with Claude SDK

4-Way Fan-out Comparison Test

Distributed Workers: Adding a Second Host

Most Panicked Moment During Operation

Summary

Building a Blog AI Auto-Comment System (1/3) — Architecture and Implementation

Overview

System Architecture

Overall Data Flow

Components

giscus Configuration

GitHub Webhook Configuration

Flask Worker Implementation

Project Structure

Core Dependencies

Webhook Endpoint

Preventing Infinite Loops

Calling Claude Code CLI

GitHub GraphQL API Integration

Input Sanitization

Audit Logging

Conclusion

Multi-Model AI Agent Team Design: Composed Architecture and 5-Team Hierarchy

Overview

Background: Why Multi-Model?

Final Team Structure

Team Details

The Infra Team Separation Decision

Model Distribution Strategy

Composed Agent Architecture

Composed Approach

Module Structure

Advantages

Expert-Definition Mapping

TLS Certificate Strategy: Cloudflare Origin CA

Relay Plugin: Agent Invocation Mechanism

backed_by Namespaces

Design Decision Log

Retrospective: What I Learned

1. “Executable Structure” over “Perfect Structure”

2. Workspace Defines Team Boundaries

3. The Value of Composed Architecture

4. Cost is Determined at Design Time

Next Steps

[LLM] How to Write Effective Prompts

Introduction

Core Principles of Good Prompts

1. Clarity

2. Provide Context

3. Role Playing

4. Specify Output Format

Prompt Patterns

Chain of Thought

Few-Shot Learning

Structured Prompts

Common Mistakes

Practical Checklist

Conclusion

The Core: Backend Adapter in `poller.py`

Declarative Management: `fleet.yaml` × `servers.yaml`

`servers.yaml` — Server Inventory

`fleet.yaml` — Worker Placement

Worker Templating System: `provision_worker.py`

`provision_worker.py` Usage