Skip to main content

​
What is Simforge?

Simforge captures your AI function calls to automatically generate evaluations. Re-run your prompts with different models, parameters, and inputs to iterate faster.
  • Capture: Wrap your AI functions to capture inputs and outputs
  • Label: Review and tag captured calls to build evaluation datasets
  • Evaluate: Re-run with different configurations to compare results
  • Iterate: Use insights from evaluations to improve your prompts

​
Quick Start

Get API Keys

Create API keys to authenticate with Simforge

TypeScript SDK

Capture AI functions in TypeScript

Python SDK

Capture AI functions in Python

Ruby SDK

Capture AI functions in Ruby

Go SDK

Capture AI functions in Go

VS Code Extension

Install the VS Code extension for IDE integration

​
How It Works

  1. Wrap Functions: Capture your AI function calls with a simple wrapper
  2. Review Captures: View captured inputs and outputs in the Labeling section
  3. Build Evaluations: Create test cases from real production data
  4. Re-run and Compare: Test different models and configurations

​
TypeScript

import { Simforge } from "@goharvest/simforge"

const simforge = new Simforge({ apiKey: process.env.SIMFORGE_API_KEY })
const myService = simforge.getFunction("my-service")

const tracedFn = myService.withSpan(async (text: string) => {
  return { result: "..." }
})

​
Python

import os
from simforge import Simforge

simforge = Simforge(api_key=os.environ["SIMFORGE_API_KEY"])
my_service = simforge.get_function("my-service")

@my_service.span()
def my_function(text: str) -> dict:
    return {"result": "..."}

​
Ruby

require "simforge"

Simforge.configure(api_key: ENV.fetch("SIMFORGE_API_KEY"))

class MyService
  include Simforge::Traceable
  simforge_function "my-service"

  simforge_span :my_method
  def my_method(text)
    { result: "..." }
  end
end

​
Go

import simforge "github.com/Project-White-Rabbit/simforge-go"

client := simforge.NewClient(os.Getenv("SIMFORGE_API_KEY"))
myService := client.GetFunction("my-service")

result, err := myService.Span(ctx, func(ctx context.Context) (any, error) {
    return map[string]any{"result": "..."}, nil
})

client.FlushTraces(5 * time.Second)

​
BAML Functions

Simforge also supports BAML for defining structured LLM functions with:
  • Type-safe inputs and outputs
  • Prompt templating
  • Provider configuration
See the Functions documentation for details on creating and managing BAML functions.