Skip to main content
The Simforge Ruby SDK captures your AI function calls to automatically generate evaluations. Re-run your prompts with different models, parameters, and inputs to iterate faster.

​
Installation

# Bundler
bundle add simforge

# Gem
gem install simforge

​
Quick Start

require "simforge"

Simforge.configure(api_key: ENV.fetch("SIMFORGE_API_KEY"))
Copy this prompt into your coding agent (tested with Cursor and Claude Code using Sonnet 4.5):
Modify existing Ruby code to add Simforge tracing.
Do NOT browse or web search. Use ONLY the API described below.

Simforge Ruby SDK (authoritative excerpt):
- Install: `gem install simforge` or `bundle add simforge`
- Init:
  require "simforge"
  Simforge.configure(api_key: ENV.fetch("SIMFORGE_API_KEY"))
- Instrumentation (ONLY allowed form):
  class MyService
    include Simforge::Traceable
    simforge_function "<trace_function_key>"

    simforge_span :method_name, type: "function"
    def method_name
      # ...
    end
  end
  (simforge_span must be placed immediately ABOVE the `def` it instruments.)
- Span types: "llm", "agent", "function", "guardrail", "handoff", "custom"
- DO NOT use a block form of simforge_span.
- DO NOT extract helper methods.

STRICT RULE:
You MUST ask me which method to instrument before making ANY code changes.
Do NOT choose a method yourself.

Task:
1) Ensure the simforge gem is added and initialization exists (Gemfile + initializer).
2) Ask me which EXISTING method should be instrumented with Simforge.
3) After I confirm the method name:
   - Add `simforge_span` directly ABOVE that method's `def`
   - Ensure the class includes `Simforge::Traceable` and has `simforge_function` set
4) Do not change method signature, behavior, or return value. Minimal diff.

Output:
- First: your question asking which method to instrument
- After confirmation: minimal diffs for Gemfile, initializer, and the method change

​
Basic Configuration

Simforge.configure(api_key: String)

# Disable tracing (functions still execute, but no spans are sent)
Simforge.configure(api_key: String, enabled: false)

​
Tracing

Include Simforge::Traceable in a class, declare the trace function key once with simforge_function, then use simforge_span above each method: 
class OrderService
  include Simforge::Traceable
  simforge_function "order-processing"

  simforge_span :process_order, type: "function"
  def process_order(order_id)
    { order_id: order_id }
  end

  simforge_span :validate_order, type: "guardrail"
  def validate_order(order_id)
    { valid: true }
  end
end

​
Using simforge_span with Explicit Key

For a single span with an explicit trace function key: 
class StandaloneService
  include Simforge::Traceable

  simforge_span :standalone_task, trace_function_key: "one-off-operation"
  def standalone_task
    "done"
  end
end

​
Automatic Nesting

Spans nest automatically based on call stack: 
class Pipeline
  include Simforge::Traceable
  simforge_function "pipeline"

  simforge_span :outer, type: "agent"
  def outer
    inner  # Becomes a child of "outer"
  end

  simforge_span :inner, type: "function"
  def inner
    # ...
  end
end

​
Span Options

Parameters:
  • method_name (required): Symbol of the method to wrap
  • trace_function_key (optional): Override class-level simforge_function
  • name (optional): Display name. Defaults to method name
  • type (optional): Span type. Defaults to "custom"
Span Types: 
SPAN_TYPES = %w[
  llm        # LLM calls
  agent      # Agent workflows
  function   # Function calls
  guardrail  # Safety checks
  handoff    # Human handoffs
  custom     # Default
]
Examples: 
class SafetyService
  include Simforge::Traceable
  simforge_function "safety-service"

  # Method name is automatically captured as span name
  simforge_span :check_safety, type: "guardrail"
  def check_safety(content)
    { safe: !content.include?("unsafe") }
  end

  # Override with name option
  simforge_span :validate_input, name: "InputValidator", type: "guardrail"
  def validate_input(input)
    { valid: !input.empty? }
  end
end

​
Error Handling

Errors are captured in the span and re-raised: 
class RiskyService
  include Simforge::Traceable
  simforge_function "risky-service"

  simforge_span :risky
  def risky
    raise "error"
  end
end

begin
  RiskyService.new.risky
rescue => e
  # Span records error and timing
end

​
Flushing Traces

Simforge.flush_traces(timeout: 30)  # Default: 30s
Traces flush automatically on process exit via at_exit hook.

​
Wrapping Third-Party Methods

Use Simforge::Traceable.wrap to trace methods on external classes: 
require "openai"

Simforge::Traceable.wrap(
  OpenAI::Client, :chat,
  trace_function_key: "openai",
  name: "Chat",
  type: "llm"
)

# Now all calls to client.chat are traced
client = OpenAI::Client.new(access_token: ENV["OPENAI_API_KEY"])
client.chat(parameters: { model: "gpt-4", messages: [...] })

​
Native Functions

Coming soon.

​
Sim Runners

Coming soon.