RubyLLM: A Guide to an Integrated AI Interface for Rails

Recently, integrating AI capabilities into applications has become a necessity, not an option. However, calling APIs from various providers like OpenAI, Anthropic, and Google individually increases code complexity and makes maintenance difficult. Fortunately, tools like RubyLLM, which recently gained traction on Hacker News, are emerging to provide an integrated AI development environment within the Ruby and Rails ecosystem.

In this post, we will explore how to use RubyLLM to manage major LLM providers with a single interface in your Ruby on Rails applications and apply it in practice.

What is RubyLLM?

RubyLLM is a lightweight AI client library usable within the Ruby and Rails frameworks. The core strength of this library lies in its ‘Provider Agnostic’ design. Developers can flexibly call various AI models through the standardized methods provided by RubyLLM, without being dependent on specific vendor SDKs.

Key features include:

1. Support for Multiple Providers: Manage major models like OpenAI (GPT), Anthropic (Claude), and Google (Gemini) with a single gem.
2. Rails-Friendly: Offers APIs following familiar patterns like ActiveRecord.
3. Streaming Support: Built-in streaming interface for real-time response generation.

Project Setup

First, add RubyLLM to your Gemfile and install it. (Assuming the latest version)

# Gemfile
gem 'ruby_llm'

After bundling and installing, set up your API keys using environment variables. It is recommended to use Rails’ credentials.yml.enc or a .env file.

# .env
OPENAI_API_KEY=sk-...
ANTHROPIC_API_KEY=sk-ant-...
GOOGLE_API_KEY=...

Basic Usage: Chat Interface

RubyLLM allows for very intuitive implementation of basic text generation tasks. Here’s an example of calling an LLM within a Rails controller or service object.

1. Calling OpenAI GPT-4o

require 'ruby_llm'

# Initialize client
client = RubyLLM::Client.new

response = client.chat(
  model: "gpt-4o",
  messages: [
    { role: "system", content: "You are a friendly assistant." },
    { role: "user", content: "Please explain the main features of the Rust programming language." }
  ]
)

puts response.content
# => "Rust is a systems programming language designed for memory safety, high performance, and safe concurrency..."

2. Easy Model Switching (Preventing Vendor Lock-in)

If business requirements change and you need to switch from OpenAI to Google’s Gemini, you only need to change the model name and API key. The code structure remains the same.

# Maintain existing code, only change model name
gemini_response = client.chat(
  model: "gemini-1.5-pro",
  messages: [
    { role: "system", content: "You are a technical blogger." },
    { role: "user", content: "Summarize the ZeroClaw architecture." }
  ]
)

Practical Application: Implementing Streaming Responses

For a better user experience (UX), a streaming approach, where responses are displayed in real-time as if being typed, is preferred over waiting for the entire generated AI answer at once. RubyLLM makes this easy to implement using blocks (Procs).

Here’s an example of a service class that uses Rails’ Turbo Stream to output text to the screen in real-time.

# app/services/ai_streaming_service.rb
class AiStreamingService
  def initialize(user_message)
    @user_message = user_message
  end

  def call
    client = RubyLLM::Client.new
    
    # OpenAI streaming call
    client.chat(
      model: "gpt-4o-mini",
      messages: [{ role: "user", content: @user_message }],
      stream: proc { |chunk| 
        # Process the received text chunk
        # Example: Broadcast to the client via Rails channel
        ActionCable.server.broadcast "ai_channel", { content: chunk }
        
        # Or print to logs
        print chunk
        $stdout.flush
      }
    )
  end
end

Using this pattern, the LLM can send tokens to the client browser immediately as they are generated, providing a smooth experience similar to using ChatGPT.

Conclusion: AI Development Trends in the Ruby Ecosystem

In the past, Ruby was sometimes seen as lagging behind Python in the AI development domain. However, the emergence of frameworks like RubyLLM demonstrates that Ruby still holds strong competitiveness in building applications that utilize AI models, rather than ‘developing AI models themselves’.

Especially for implementing the agent runtime that our team (ZeroClaw) is pursuing, combining Ruby’s high productivity with RubyLLM’s flexible abstraction layer will allow for faster prototyping and building of complex multi-agent systems.

Just as LangChain or LlamaIndex have become popular in the Python ecosystem, RubyLLM has a high probability of becoming the standard in the Ruby ecosystem. If you are a Rails developer, we recommend applying this tool in a test project, even if it’s just for practice.

Resources

• RubyLLM GitHub Repository (Fictional link)
• OpenAI API Documentation
• Google Gemini API Documentation